Skip to content

docs(verify-pyprocar): add a compare helper and fix map drift at c13166ce - #315

Open
lllangWV wants to merge 1 commit into
devfrom
chore/verify-skill-maintain
Open

lllangWV wants to merge 1 commit into
devfrom
chore/verify-skill-maintain

Conversation

@lllangWV

@lllangWV lllangWV commented Oct 6, 2026

Copy link
Copy Markdown
Member

Summary

This PR maintains the verify-pyprocar skill at dev c13166ce. Each feature file now has an expected status for every driver step, and the new verify.sh compare checks runs against those statuses. The drivers now cover the public entry points that the map did not drive. The map no longer says that object-API energies are Fermi-shifted. The proven end states are re-pinned to c13166ce.

Only files in .claude/skills/verify-pyprocar/ change. No product code changes.

What changed

  1. verify.sh compare <run-dir>... (scripts/compare.py). It finds the driver by the bytes of evidence/driver.py. It then checks summary.json, side_effects.txt and the exit code against the ## Expected step status table in the feature files. Each row is ok, or known-defect with an error type and a crash site. It exits 1 when any run deviates.
  2. Stable crash sites. verify_steps recorded where from the checkout directory name. In a worktree it gave plotter/bs_plot.py:986, a matplotlib frame, or a path that held the run directory. Now it is pyprocar/<path>:<line>, else driver.py:<line>.
  3. New driver steps:
    • bands.py: plot_overlay with an array, plot_quiver given ebs.bands, legacy overlay_orbitals.
    • dos.py: the seven dosplot modes that were not driven.
    • bs2d.py: plot(show_brillouin_zone=True).
    • fermi3d.py: FermiHandler save_3d and plot_fermi_cross_section_box_widget.
    • fermi2d.py: plain_bands.
    • parsers.py: Elk, Abinit and QE non-collinear bands.
    • utilities.py: spin_asymmetry, download_from_hf with a str path, and scripts/procar.py cat, filter, bandgap and generate2dkmesh, run in-process.
  4. Doc drift fixed (each item was checked against source or a live run):

Verification

All runs are in data/verify-runs/ of the main checkout, driven from a worktree at c13166ce plus this change.

  1. The whole-map loop from SKILL.md, tag map232618:
    $ verify.sh compare data/verify-runs/*-map232618-*
    matches   20261005-232618-map232618-bands
    ...
    matches   20261005-232726-map232618-ref_spin_ibz
    0 of 11 run(s) deviate
    
  2. A re-drive with the final harness (*-final-{bands,utilities,dos}, evidence written at 23:28) gives 0 of 3 run(s) deviate.
  3. compare fails when it should. I planted one violation in each of 7 scratch copies of the bands run. The violations were an ebs.pkl side effect, a failed ok step, a fixed defect, a moved crash site, an unmapped step, a wrong exit code and an edited driver. The output was 7 of 8 run(s) deviate, and the unchanged copy matched.
  4. Gates:
    • basedpyright --project pyrightconfig.json: 0 errors, 0 warnings.
    • ruff_new_violations.py check|format origin/dev: 9 changed Python files, ok.
    • pytest tests/test_verify_fixtures.py tests/test_fixture_copies.py: 173 passed, 2 skipped.
  5. I opened the new PNGs and checked them:
    • obj_bs2d_bz.png shows the hexagonal prism.
    • legacy_box_widget_slice.png shows the closed orbits around Γ.
    • legacy_stack_species.png shows the stacked DOS under the total.
    • obj_plain.png shows E_F near +5 eV.

Product gaps found (not fixed here)

The map records these as known-defect rows. Each one reproduces in the utilities or dos run.

  • download_from_hf(relpath, output_path="."): TypeError at pyprocar/utils/download_examples.py:143. The installed pyprocar-download CLI hits the same error for every relpath, prints Download failed and exits 1.
  • python scripts/procar.py bandgap PROCAR OUTCAR vasp 5.3017: ValueError: Invalid code: 5.3017. Its arguments go in the wrong order at scripts/procar.py:124.
  • python scripts/procar.py generate2dkmesh -0.5 -0.5 0.5 0.5 0 5 4: TypeError at pyprocar/scripts/scriptKmesh2D.py:42. Its arguments have no type=.
  • dosplot(mode="stack") without items: IndexError at pyprocar/scripts/scriptDosplot.py:427.
  • pyprocar.spin_asymmetry is still exported, but it always raises NotImplementedError.

Not verified

  • The download of data/codes.zip by $H fetch data/codes. The source suggests that it extracts to data/codes/codes/. I did not download it.
  • procar.py kpath defaults -with_time_reversal to False, but kpath() defaults it to True. This is from source only.
  • The shared data/ store has 2223 writable fixture paths, because the fixtures predate feat(verify-pyprocar): make fetched fixtures read-only and keep verify.sh inside data/ #303. I did not lock them. The main checkout is on an older dev.



Add `verify.sh compare`, which checks each run against a new
"Expected step status" table in every feature file. Make the recorded
crash site the same in every worktree and run. Drive the map's missing
entry points (every dosplot mode, plot_quiver and overlay_orbitals,
the zone in BS2DPlotter, FermiHandler save_3d and the box widget, Elk,
Abinit and QE non-collinear parsers, spin_asymmetry, download_from_hf
and the procar.py CLI). Correct the claims that object-API energies
are Fermi-shifted and re-pin the proven end states to c13166c.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant